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DETAILED ACTION 

Response to Amendment 

1 . This Office Action is response to Applicant's remarks received on March 25, 
2008. Claims 1-15 remain pending. 

Drawings 

2. Figure 1 is objected to as failing to comply with 37 CFR 1 .84(p)(5) because they 
include the following reference character(s) not mentioned in the description: 16. 

Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in 
reply to the Office action to avoid abandonment of the application. Any amended 
replacement drawing sheet should include all of the figures appearing on the immediate 
prior version of the sheet, even if only one figure is being amended. Each drawing sheet 
submitted after the filing date of an application must be labeled in the top margin as 
either "Replacement Sheet" or "New Sheet" pursuant to 37 CFR 1.121 (d). If the 
changes are not accepted by the examiner, the applicant will be notified and informed of 
any required corrective action in the next Office action. The objection to the drawings 
will not be held in abeyance. 
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Specification 

3. The previous specification objections have been withdrawn in light of Applicant's 
amendment. 

Claim Suggestions 

4. The previous claim suggestion has been withdrawn in light of Applicant's 
amendment. 

Claim Objections 

5. The previous claim objections have been withdrawn in light of Applicant's 
amendment. 

Claim Rejections - 35 USC § 102 

6. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 
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A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351 (a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 



7. Claims 1 , 2, 8-1 2 and 1 5 are rejected under 35 U.S.C. 1 02(e) as being 
anticipated by Watanabe (US 2003/0048926). 

Regarding claim 1, Watanabe discloses a method for processing video images 
to detect an event of interest ("surveillance system and surveillance method and 
surveillance program for automatically detecting a person matching prescribed 
conditions from surveillance images" at paragraph 0002, line 1), comprising: 

receiving a video signal representing the video images to be processed (figure 
15, numeral 10); 

extracting at least one point feature from the video signal (figure 16, numeral 12); 

tracking the position and movement of the at least one point feature within the 
video images to generate a corresponding at least one track, each of said at least one 
track representing a corresponding point feature (figure 16, numeral 13); 

using an iterative learning process to derive a normal pattern of behavior for each 
track (figure 16, numeral 64); 

comparing present behavior of the at least one track to the respective normal 
pattern of behavior (figure 15, numeral 81); and 
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in response to the present behavior falling outside the normal pattern of behavior, 
generating an alarm signal ("information for a theft suspect" at paragraph 0149, line 7). 

Regarding claim 2, Watanabe discloses a method wherein the alarm signal 
causes at least one of the following effects: 

draw the attention of the operator ("sending information about a specific person 
to be detected by one particular client section 60" at paragraph 0163, line 1); 

place an index mark at the appropriate place in recorded video data; and 

trigger selective recording of video data. 

Regarding claim 8, Watanabe discloses a method wherein the comparison 
process acts to classify as normal behavior a track adjacent or near a cell (figure 9) 
which is above the occupancy threshold, despite the track appearing in a cell below the 
occupancy threshold, where one cell is considered to be near another if the distance 
between them is below a predetermined distance threshold ("whether or not the 
characteristics of the human region thus detected match the characteristics of the 
specific person" at paragraph 01 1 1 , line 5). 

Regarding claim 9, Watanabe discloses a method wherein abnormal tracks are 
filtered, whereby an active alarm signal is generated in response to an abnormal track 
which resembles a number of other abnormal tracks ("detects the suspect from the 
surveillance images" at paragraph 0149, line 10), in terms of at least one of position, 
velocity and time. 

Regarding claim 10, Watanabe discloses a method wherein abnormal tracks are 
filtered, whereby an active alarm signal is generated in response only to an abnormal 
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track which has been classified as abnormal on a predetermined number of occasions 
(i.e. once). 

Regarding claim 11, Watanabe discloses a method wherein abnormal tracks are 
filtered, whereby an active alarm signal is generated in response only to a track being 
classified as abnormal for the first time ("inputs information about a specific person (for 
example, a person who has conducted suspicious actions)" at paragraph 0160, line 21). 

Regarding claim 12, Watanabe discloses a method wherein abnormal tracks are 
filtered, whereby an active alarm signal is generated only in response to a filtered 
version of the classification rising above a predetermined threshold value ("If it is judged 
that the face image in the specific person table 24 does match" at paragraph 01 12, line 
1)- 

Regarding claim 15, Watanabe discloses an apparatus for processing video 
images to detect an event of interest ("surveillance system and surveillance method and 
surveillance program for automatically detecting a person matching prescribed 
conditions from surveillance images" at paragraph 0002, line 1), comprising: 

a source of video images, which produces a video signal representing the video 
images to be processed (figure 15, numeral 10); 

a feature extraction device that receives the video signal, and produces data 
representing at least one point feature detected within the image (figure 16, numeral 
12); 

a feature tracking device that receives the data representing said at least one 
point feature, and produces data representing a track that is representative of position 
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and speed of each of said at least one point feature within the image (figure 16, numeral 
13); 

a learning device that receives the data representing tracks of said at least one 
point feature, and produces a signal representing a range of behavior considered 
normal by the learning device, in response to operation of a learning process on the 
data representing the tracks (figure 16, numeral 64); 

a classification device that receives both the signal representing the normal 
range of behavior of the tracks and the data representing the tracks, and is adapted to 
compare the signal and the data and to issue a normal/abnormal signal in accordance 
with the outcome of such comparison (figure 15, numeral 81); and 

an alarm generation device that receives the normal/abnormal signal and 
generates at least one active alarm signal in response to the normal/abnormal signal 
indicating abnormal behavior of at least one track ("information for a theft suspect" at 
paragraph 0149, line 7). 



Claim Rejections - 35 USC § 103 



8. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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9. Claims 3, 4 and 7 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over the combination of Watanabe and Schwerdt et al. ("Visual Recognition of 
Emotional States", ICMI). 

Regarding claims 3 and 4, Watanabe discloses a method wherein the learning 
process accumulates data representing the behavior of the track(s) over a period of 
time using four dimensions representing x-position, y-position, x-velocity and y-velocity, 
of the track(s) within the video image (figure 9) wherein the learn behavior stage 
segregates the tracks according to a velocity threshold; tracks moving at a velocity 
below the velocity threshold are considered stationary (figure 9, "Stay") while tracks 
moving at a velocity in excess of the velocity threshold are considered mobile (figure 9, 
"Move"); data concerning the mobile tracks is stored, data concerning the stationary 
tracks is stored in a two-dimension histogram (figure 6) representing x-position and y- 
position within the video image. 

Watanabe does not disclose that the data concerning the mobile tracks is stored 
in a four-dimensional histogram. 

Schwerdt et al. discloses a method in the same field of endeavor of object 
tracking and recognition that accumulates data representing the behavior of the mobile 
track(s) over a period of time in a four-dimensional histogram ("four dimensional 
histogram" at section 3.2, paragraph 4, line 1). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to utilize the 4-D histogram of Schwerdt et al. to keep track of the data of 
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Watanabe "because the memory space requirements are lower" (Schwerdt et al. at 
section 3.2, paragraph 6, line 8). 

Regarding claim 7, Schwerdt et al. discloses a method wherein the comparison 
process classifies a track according to a comparison of the frequency of occupation of 
the corresponding histogram cell with an occupancy threshold (Table 2). 

10. Claim 13 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Watanabe and Kamin (US 4,198,653). 

Watanabe discloses the elements of claim 1 as described in the 102 rejection 

above. 

Watanabe does not disclose that subsequent active alarm signals are inhibited 
for a predetermined time interval after a first active alarm signal has been produced. 

Kamin discloses a method in the same field of endeavor of video tracking 
wherein subsequent active alarm signals are inhibited for a predetermined time interval 
after a first active alarm signal has been produced ("first alarm pulse A' (Fig. 3d), which 
originates from field 2 and which would normally result in a spurious alarm, is sppressed 
at the right time" at col. 4, line 24). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to utilize the delay of Kamin in the alarm generation device of Watanabe 
to provide "high sensitivity with respect to events which are relevant to a genuine alarm" 
(Kamin at col. 1, line 59). 
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1 1 . Claim 14 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Watanabe and Wyschogrod et al. (US 5,374,932). 

Watanabe discloses the elements of claim 1 as described in the 102 rejection above. 

Watanabe does not disclose that subsequent active alarm signals are inhibited if 
caused by an abnormal track within a predetermined distance of another track which 
has previously generated an alarm. 

Wyschogrod et al. discloses a method in the same field of endeavor of video 
surveillance wherein subsequent active alarm signals are inhibited if caused by an 
abnormal track within a predetermined distance of another track which has previously 
generated an alarm ("alerts to suppressed if the target on approach is projected onto 
more than one runway" at col. 106, line 23). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to utilize the alert suppression in the alarm generation device of Watanabe 
to prevent alert generation for false targets. 

12. Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Watanabe and Schwerdt et al. as applied to claim 3 above, and further 
in view of Zuniga (US 5,546,474). 

The Watanabe and Schwerdt et al. combination discloses the elements of claim 3 as 
described in the 103 rejection above. 
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The Watanabe and Schwerdt et al. combination does not disclose that a cell size 
of the histogram varies in accordance with a measured speed in the image of each 
respective track. 

Zuniga discloses a method in the same field of endeavor of region detection 
wherein a cell size ("size of a cell can vary" at col. 10, line 14) of the histogram varies in 
accordance with a measured speed in the image of each respective track ("moment of 
intertia" at col. 10, line 9). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to utilize the cell varying of Zuniga in the histogram of the Watanabe and 
Schwerdt et al. combination to allow the user to evaluate a "performance/quality 
tradeoff' (Zuniga at col. 10, line 15). 

13. Claim 6 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Watanabe and Schwerdt et al. as applied to claim 3 above, and further 
in view of Jepson et al. (US 7,058,205). 

The Watanabe and Schwerdt et al. combination discloses the elements of claim 
3 as described in the 103 rejection above. 

The Watanabe and Schwerdt et al. combination does not disclose that the 
histogram is periodically de-weighted in order to bias the result of the learning process 
towards more recent events. 

Jepson et al. discloses a method in the same field of endeavor of motion tracking 
wherein the histogram is periodically de-weighted in order to bias the result of the 
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learning process towards more recent events ("up-weight the more recent frames" at 
col. 3, line 19). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to utilize the weight varying of Jepson et al. to the histogram of the 
Watanabe and Schwerdt et al. combination such that a "optimal motion estimation is 
achieved" (Jepson et al. at col. 3, line 12). 



Response to Arguments 

Summary of Remarks (@ response page labeled 18): The Watanabe reference 
does not implement a learning function that determines normal behavior of a track. 

Examiner's Response: As the Watanabe reference discloses monitoring behavior 
of people, which includes product tendencies and whether or not they are a high 
spender, if follows that the system learns the repeated normal behavior of people. 

Summary of Remarks (@ response page labeled 19): The Watanabe reference 
does not classify behavior as normal or abnormal. 

Examiner's Response: The Watanabe reference detects specific persons based 
on behavior tables. It sometimes happens that the detected person in the surveillance 
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image does not match the particular characteristics of the specific person being 
searched for, even though it was determined that they were a match. As such, a 
detected person, who may be classified as a high spender, can be flagged as a theft 
suspect, which would be abnormal behavior for that detected person. 

Summary of Remarks (@ response page labeled 19): The Watanabe reference 
does not disclose generating an alarm signal to warn of abnormal behavior. 

Examiner's Response: The Watanabe reference outputs a display or audio 
warning in the previously described scenario, which amounts to an alarm signal of 
abnormal behavior. 

Summary of Remarks (@ response page labeled 23): The Schwerdt et al. 
reference does not disclose learning normal behavior. 

Examiner's Response: The Schwerdt et al. reference was not utilized for the 
specifics of the learning process, but rather as a teaching of how 4-dimensional 
histograms can be used in a tracking environment. 

Summary of Remarks (@ response page labeled 25): The Schwerdt et al. 
reference does not disclose an occupancy threshold and does not relate to "whether 
observed behavior is normal or abnormal". 
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Examiner's Response: The Schwerdt et al. reference contains an occupancy 
threshold which corresponds to the number of cells that occupy the histogram and 
thereby affects the reliability of the classification. Furthermore, the Schwerdt et al. 
reference need not teach classifying observed behavior, as it is utilized for teachings in 
the classification comparison process in general. 

Summary of Remarks (@ response page labeled 25): The Kamin reference 
"attempts to block aN alarm indications - not just ones following a first active alarm". 

Examiner's Response: The first alarm pulse as cited in the rejection is an active 
alarm that has been produced and alarm pulses subsequent to this during a certain time 
period are inhibited, as required by the claim. 

Summary of Remarks (@ response page labeled 26): The Wyschogrod et al. 
reference does not disclose a track that has previously generated an alarm. 

Examiner's Response: The arriving target generates an alert as soon as it is 
determined that it is projected to land. If an arriving target becomes ambiguously 
projected subsequent to initiating the landing alert, alerts can be suppressed. 
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Summary of Remarks (@ response page labeled 21 ): None of the cited 
references disclose the elements of claims 5-12. 

Examiner's Response: The Examiner has cited references for the elements of 
claims 5-12 in the rejections above which address the elements of the claims. As 
Applicant has not pointed out any specifics with regard to the deficiencies of the cited 
references, the Examiner will not address these arguments any further. 



Conclusion 

14. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 
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1 5. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to KATRINA FUJITA whose telephone number is 
(571)270-1574. The examiner can normally be reached on M-Th 8-5:30pm, F 8- 
4:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Vikkram Bali can be reached on (571) 272-7415. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Katrina Fujita/ 
Examiner, Art Unit 2624 



/Vikkram Bali/ 

Supervisory Patent Examiner, Art Unit 2624 



